Information acquisition apparatus

ABSTRACT

When a plurality of objects having RFID tags attached are to be read, an image sensing unit performs image sensing, and at the same time an RFID reader reads an RFID tag within the image sensing range. A characteristic quantity (external appearance model) of an object obtained from a database on the basis of ID data of the RFID tag is compared with a characteristic quantity obtained by image processing of the sensed image, thereby acquiring information unique to the object. The object is discriminably specified by superimposing the acquired unique information on the object in the sensed image.

FIELD OF THE INVENTION

The present invention relates to a technique which acquires informationof an object in a sensed image and discriminably specifies the object byusing the radio frequency identification (RFID) technique and so on.

BACKGROUND OF THE INVENTION

The radio frequency identification (RFID) technique identifies an RFIDtag (or an IC tag, IC card, or the like) by externally reading data suchas an identifier of the tag in a non-contact manner by using anelectromagnetic wave. Various application fields are possible byembedding a writable memory in the tag or combining the tag with asensor. The RFID tags are roughly classified into an active tag(equipped with a power supply) and a passive tag (equipped with no powersupply) in accordance with whether a tag has a power supply (battery).

With the advance of the downsizing technique, the passive RFID tag whichoperates by an electromagnetic wave supplied from an external apparatusis rapidly expanding its application fields in recent years. Well-knownexamples are a non-contact IC card type pass and entrance card.

It is also possible to adhere the RFID tag contained in a seal to anarticle, and automatically detect the existence and stored informationof the RFID tag by an RFID reader installed in a location where thearticle is placed or near the entrance/exit of a room, therebypreventing unauthorized takeout or theft of the article. In addition,there is a plan which uses this RFID tag in efficient article managementin all stages (production, distribution, and sale) of supply chainmanagement, as the next-generation article identification techniqueinstead of barcodes.

The transmitting power of many passive RFID tags described above isweak, so these tags can be read only in the vicinity (about a few cm toa few tens of cm) of the RFID reader. However, active RFID tags can beread from a relatively remote place (to about a few m) because thetransmitting power is high.

When the RFID tags are attached to many articles, a plurality of RFIDtags exist within the reading range of the RFID reader. In theconventional general system, pieces of information of a plurality ofRFID tags existing within the reading range of the RFID reader can besimultaneously read, but it is impossible to determine to which objecteach read information belongs.

To solve this problem, Japanese Patent Laid-Open No. 10-229351 (priorart 1) discloses a system in which a specific RFID tag is irradiatedwith a directional optical signal, and only the RFID tag having receivedthe optical signal transmits information on an electromagnetic wave to areader.

Also, Japanese Patent Laid-Open No. 2000-227948 (prior art 2) disclosesa system in which an RFID tag emits light, and the position of this RFIDtag is detected on the basis of an image sensed by an image sensor. Thismethod makes it possible to relate information of each RFID tag to itsposition information, and determine one of a plurality of RFID tags fromwhich information is being read.

In this system, however, the position of an RFID tag is detected byusing light. Therefore, if an RFID tag is located in a hidden position,it is impossible to detect the position of this RFID tag and specify it.Also, a function of receiving or emitting light must be added to an RFIDtag, and this extremely complicates the structure of the RFID tag. Toattach RFID tags to all articles, each individual RFID tag is desirablysimple in arrangement and inexpensive.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the aboveproblems, and has as its object to specify an object in a sensed imageso that the object can be discriminated by its predeterminedinformation.

It is another object of the present invention to make an object in asensed image visually readily understandable by a human and widelyapplicable.

It is still another object of the present invention to make it possibleto detect an object and determine its existing position even if a tag islocated in an invisible position or the object is partially obstructed.

It is still another object of the present invention to reduce the loadof the object detecting process or the existing position determinationprocess.

To achieve the above objects, according to an aspect of the presentinvention, there is provided an information acquisition apparatus havinga sensing device configured to sense an object, and a reading deviceconfigured to read information of a tag attached to the object,comprising: an extracting device configured to extract informationconcerning a characteristic of an object in an image sensed by thesensing device; and a discrimination device configured to discriminate aposition of the object corresponding to the information of the tag readby the reading device, on the basis of the information concerning thecharacteristic of the object in the sensed image extracted by theextracting device and the information of the tag read by the readingdevice.

According to another aspect of the present invention, there is providedan information acquisition apparatus processing method comprising: anextraction step of extracting information concerning a characteristic ofan object in a sensed image; an identification step of identifyinginformation of a tag attached to the object in the sensed image; and adiscrimination step of discriminating a position of the objectcorresponding to the information of the tag identified in theidentification step, on the basis of the information concerning thecharacteristic of the object in the sensed image extracted in theextraction step, and the information of the tag identified in theidentification step.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an object information acquisition systemaccording to the first embodiment;

FIG. 2 is a flowchart of the object information acquisition systemaccording to the first embodiment;

FIG. 3 is a view showing a practical operation example of the objectinformation acquisition system according to the first embodiment;

FIG. 4 shows information stored in a database according to the firstembodiment;

FIG. 5 is a view showing a display example on a display unit accordingto the first embodiment;

FIG. 6 is a view showing switching of information display according tothe first embodiment;

FIG. 7 is a view showing a sensed image in the object informationacquisition system according to the first embodiment;

FIG. 8 is a view showing a display example on the display unit of theobject information acquisition system according to the first embodiment;

FIG. 9 shows information stored in the database according to the firstembodiment;

FIG. 10 is a flowchart of an object information acquisition systemaccording to the second embodiment;

FIGS. 11A and 11B are views showing practical operation examples of theobject information acquisition system according to the secondembodiment;

FIG. 12 is a table showing a practical example of a correspondence tableof the radio field intensity from an RFID tag and the enlargement orreduction ratio of the characteristic quantity of an object;

FIG. 13 is a block diagram of an object information acquisition systemaccording to the third embodiment;

FIG. 14 is a block diagram showing an example of the arrangement of anobject information acquisition system according to the fourthembodiment;

FIG. 15 is a view showing an example of the arrangement of a directionalantenna 521 having controllable characteristics;

FIG. 16 is a view showing a practical example of the correspondence ofthe zooming magnification of image sensing as one sensing parameter tothe feeding range of the antenna as one RFID read parameter;

FIG. 17 is a view showing a practical example of the correspondence ofthe focal distance (F-number) of image sensing as one sensing parameterto the output level of the antenna as one RFID read parameter;

FIG. 18 is a flowchart showing the control sequence of a parametercontroller 530 according to the fourth embodiment; and

FIG. 19 is a block diagram showing an example of the arrangement of anobject authentication apparatus according to the fifth embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the present invention will now be described indetail in accordance with the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram of an object information acquisition systemaccording to the first embodiment. FIG. 2 is a flowchart of processingin the object information acquisition system according to the firstembodiment. FIG. 1 shows an object information acquisition system 100 inwhich constituent elements are integrated. However, it is notparticularly necessary to integrate these constituent elements, and thesystem may also communicate, where necessary, with a constituent elementinstalled in another place through wire or by wireless. As shown in FIG.1, the object information acquisition system 100 includes the followingconstituent elements.

An image sensing unit 101 senses objects 10 within the range of aviewing angle 110. An RFID reader 102 has a directional antenna forcommunicating only with RFID tags in a specific direction by radio. Arange (direction range of the antenna) 111 in which a radio fieldemitted from the antenna radiates has directivity equivalent to theviewing angle 110 of the image sensing unit 101. By this directivity ofthe antenna, only RFID tags existing within the range of a sensed imageare read.

An image processor 103 receives data of the sensed image from the imagesensing unit 101, and automatically extracts the characteristicquantities, such as the edge, the color, the luminance, the texture, thepresence/absence of a motion, the shape, and the distance betweencharacteristic points, of the sensed image, by using a predeterminedimage processing algorithm.

A database 104 prestores IDs unique to individual objects, and uniqueinformation (e.g., the name, price, model number, or image information)related to the objects. The database 104 also prestores thecharacteristic quantities of the external appearance, e.g., the edge,the color, the luminance, the texture, the presence/absence of a motion,the shape, and the distance between characteristic points, of an object.The characteristic quantities are prepared by sensing an image of eachof various objects in various directions, and modeling thecharacteristics of the outer appearance of the object.

A characteristic quantity comparator 105 acquires, from the database104, only information corresponding to ID data of an RFID tag 20 read bythe RFID reader 102. The acquired information contains characteristicquantity information of an object to which the RFID tag is attached, andinformation (e.g., the name) unique to the object. The characteristicquantity comparator 105 compares the characteristic quantities of anobject acquired from the database 104 with the characteristic quantitiesof a sensed image acquired by the image processor 103, and determinesthe position of the object in the sensed image. The characteristicquantity comparison sequence is as follows. That is, the characteristicquantities, such as the edge, the color, the luminance, the texture, thepresence/absence of a motion, the shape, and the distance betweencharacteristic points, unique to the object acquired from the database104 are compared with the characteristic quantities, such as the edge,the color, the luminance, the texture, the presence/absence of a motion,the shape, and the distance between characteristic points, of the sensedimage obtained by the image processor 103, and a most matching portionis checked. If a matching portion is found, the ID data of the objectand the position in the image are stored so that they correspond to eachother. If a plurality of ID data is read by the RFID reader 102, theabove sequence is repeated the same number of times as the number of theread ID data. As a consequence, a plurality of objects to which RFIDtags are attached can be discriminated in the image. Note that if aplurality of ID data is read, information may also be simultaneouslyacquired from the database 104.

A display unit 106 displays, to the user, image data, informationindicating the correspondence of ID data of an object with the positionin an image, and information unique to the object, by methods suited tothe individual data. That is, the display unit 106 displays a sensedimage, and, in a position of the sensed image where an object exists,displays information corresponding to the ID data by characters, or byconverting the information into an icon or graph. If a plurality ofpieces of information is stored as they correspond to the ID data of theread RFID tag, the user may also freely change the contents ofinformation to be displayed, by using an input device (not shown).

With this arrangement, it is possible to discriminate between objects towhich RFID tags are attached, without adding any special function tothese RFID tags. An example of an operation of determining the positionsof a plurality of objects according to the first embodiment will bedescribed below with reference to FIGS. 2, 3, 4, and 5.

FIG. 3 is a view showing a form in which an external database 104 isaccessed by wired or wireless communication 120. An object informationacquisition system 200 shown in FIG. 3 includes an image sensing unit101, RFID reader 102, image processor 103, characteristic quantitycomparator 105, and display unit 106.

Note that although one function is placed outside the system 200, thewhole system is similar to FIG. 1. In front of the object informationacquisition system 200, objects 11 a and 12 a having RFID tags 20attached and different in external appearance exist.

FIG. 4 is a view showing an example of data of objects prestored in thedatabase 104. In this example, the ID data and characteristic quantities(external appearance information of an object) of each object arestored. In addition, the name and the date of manufacture are recordedas examples of data unique to the object. To simplify the explanation,the characteristic quantities of, e.g., Objectol are “black, cube”. Inpractice, however, detailed data which indicates, by numerical values,the edge, the color, the luminance, the texture, the presence/absence ofa motion, the shape, the distance between characteristic points, and thelike, is used.

In the object information acquisition system 200, when the image sensingunit 101 senses image (step S10 in FIG. 2), the RFID reader 102 readsinformation (ID data) of the two RFID tags 20 in substantially the samerange (step S30). Data of the characteristic quantities of objectscorresponding to the ID data read from the RFID tags 20 and information(in this case, information of the name and the date of manufacture)unique to the objects are acquired from the database 104 (step S40).

In the first embodiment, the object information acquisition system 200has a wired or wireless communication function in order to communicatewith the database 104 installed outside. In the example shown in FIG. 3,ID “00001” and ID “00002” of the RFID tags 20 are read by the RFIDreader 102, so unique information and characteristic quantity data of ID“00001” and ID “00002” are retrieved from the database 104. Althoughinformation of ID “00003” is also stored in the database 104, thisinformation is not acquired because the corresponding ID data is notread by the RFID reader 102.

The image processor 103 processes the sensed image by a predeterminedalgorithm to extract the characteristic quantities of the image (stepS20). The characteristic quantity comparator 105 compares thecharacteristics extracted from the sensed image by the image processor103 with the characteristic quantities retrieved from the database 104(step S50), thereby determining the positions of the objects in theimage (step S60). In the example shown in FIG. 3, the characteristicquantity data of ID “00001” indicates that a black cube is an object ofID “00001”, so it is possible to determine that a portion indicating thecharacteristics “black, cube” in the whole image is a portion where theobject of ID “00001” exists. This information of the position where theobject exists in the image is recorded so as to correspond to the IDdata (“00001”) of the object. Subsequently, when the characteristicquantities of the sensed image are similarly checked on the basis of thecharacteristic quantity data “white, circular column” of ID “00002”, itis also possible to determine a portion of an object of ID “00002”. Theposition information of the object in the image and the ID data arestored together for ID “00002” as well.

By the above processing, the object information acquisition system 200can acquire the ID data of each object, the information unique to theobject, and the information indicating the correspondence of the ID dataof the object with its position in the image. The display unit 106displays these pieces of information together to present informationdisplay readily seen by a human (step S70).

That is, as shown in FIG. 5, information 11 c and information 12 c ofobjects 11 b and 12 b are superimposed on portions where these objectsexist in the sensed image. In this example, pieces of information(Object01, Object02) of the names of the objects are superimposed. Withthis display, the user can easily recognize to which object theinformation read from the RFID tag belongs.

In the first embodiment, information of the date of manufacture can alsobe retrieved from the database 104, so this date-of-manufactureinformation may also be superimposed on the display. It is also possibleto display a plurality of pieces of information at the same time or byswitching them. FIG. 6 is a view showing an example in which informationdisplay 11 d is switched to another. For this switching, an inputportion for designating switching is necessary. In the preferredexample, pieces of information of a specific object in an image can beswitched by using a pointing device 115. This pointing device can be anyof a mouse, track pad, track ball, and touch panel, i.e., the type ofthe device is not limited. As another form of information display, it isalso possible to prepare icons, graphs, or computer graphics in thedatabase, and display them. By the use of the pointing device and/oricons or the like, it is possible to provide an object informationacquisition system which is visually readily understandable and widelyapplicable.

In the first embodiment, even when a plurality of simple RFID tags issimultaneously read, it is possible to determine to which object eachread information belongs.

In an actual operation, an object rarely independently exists in animage without being obstructed by anything. However, it is possible todetect the position of even an object slightly obstructed by anotherobject or by a surrounding irrelevant article. That is, if even aportion of an object exists in a sensed image, the characteristicquantities of the portion are compared with those of an object obtainedfrom the database 104. If the characteristic quantities are partiallysimilar, it is determined that the object exits in this portion. In thismanner, even when an RFID tag itself is completely invisible from thereader, the position of the object can be determined. That is, in theconventional technique which determines a position by using light, anRFID tag must be located in a position visible from the reader. In thefirst embodiment, however, it is possible to read an RFID tagintentionally located in a hidden position, or to read an RFID tag in adirection other than a surface to which the tag is attached.

FIGS. 7, 8, and 9 are views showing the way an object partiallyobstructed by another object is detected. FIG. 7 is a view showing thestate in which objects shot by the image sensing unit 101 are displayedon the display unit 106. The RFID reader 102 reads RFID tags (not shown)attached to two objects 210 and 220. The RFID tag of the object 210 hasID data “00100”, and the RFID tag of the object 220 has ID data “00101”.The characteristic quantities of objects corresponding to these ID dataand data (name data) unique to these objects are read out from thedatabase 104 (FIG. 9). The characteristic quantities of the object 210are “black, circular column”. Therefore, the position of the object inthe image can be simply determined by comparison with characteristicquantities obtained by processing the shot image.

On the other hand, the object 220 is partially hidden behind the object210, so characteristic quantities “white, circular column+cube” do notcompletely match the characteristic quantities obtained by the imageprocessor 103. Although a portion is hidden in the image, however, acharacteristic structure, i.e., a connected structure of a circularcolumn and cube, of the object 220 is clearly seen. In addition, anotherportion of the image matches the characteristic quantities of the object220. Therefore, it is possible to determine at a high probability thatthis portion is the object 220. FIG. 8 is a view showing an imagedisplayed on the display unit 106 as a result of the comparison of thecharacteristic quantities described above.

Furthermore, there is a case in which an object is completely obstructedby something and does not appear in a shot image at all, but an RFID tagcan be read. In this case, the position of the object in the imagecannot be specified, so the display unit 106 displays a messageindicating that the position in the image cannot be determined, andinformation unique to the object. By this display, the user canunderstand that the object exists in an invisible portion of the image.

From the viewpoint of image processing as well, comparison with preparedcharacteristic quantities can be performed extremely easily. That is,when characteristic quantities are compared only by image processing,all stored model images must be compared with a sensed image. In thefirst embodiment, however, objects existing in an image can be limitedby ID data read from RFID tags. This makes it possible to narrow downcharacteristic quantities to be used in comparison to minimum necessaryquantities. As a consequence, the load of image processing can begreatly reduced. In addition, if matching has failed when performed onlyby image processing, it is determined that there is no object. Forexample, if an object is hidden behind a foreground, it is difficult, byimage processing alone, not only to specify the position of the object,but also to determine the existence of the object. In the firstembodiment, however, the existence of an object can be obtained by anRFID tag. Therefore, even when an object cannot be specified in animage, the existence of the object can be displayed.

Second Embodiment

The second embodiment of the present invention will be described indetail below with reference to the accompanying drawings.

FIGS. 10, 11A, 11B, and 12 are views showing the second embodiment. Asthe second embodiment, a process of estimating the distance between anobject information acquisition system and an RFID tag from the intensityof a received radio field from the RFID tag, estimating the size of anobject in an image, and performing matching of the characteristicquantities of the object will be explained below.

An object information acquisition system of the second embodiment has acharacteristic quantity adjusting unit (not shown) in addition to theobject information acquisition system 100 (200) of the first embodiment.

FIG. 10 shows the flow of processing in the second embodiment.

Sensing (step S11), extraction of characteristic quantities by imageprocessing (step S21), read of RFID tags (step S31), and acquisition ofcharacteristic quantities of objects from a database (step S41) are thesame as in FIG. 2. Different processes are as follows. Thecharacteristic quantity adjusting unit obtains the enlargement orreduction ratio of the characteristic quantities of an object from theradio field intensity when ID data of an RFID tag is read (step S51),and the characteristic quantities are enlarged or reduced by thisenlargement or reduction ratio (step S61). In addition, a characteristicquantity comparator obtains the position of the object in the sensedimage by comparing the enlarged or reduced characteristic quantities ofthe object with the characteristic quantities of the image (step S81),and displays the position on a display unit 106 (step S91).

The enlargement or reduction ratio is obtained by checking changes inradio field intensity and changes in size of an object in an imagebeforehand, and forming and storing a correspondence table. FIG. 12shows an example of the correspondence table for enlarging or reducingthe characteristic quantities of an object. A database 104 storescharacteristic quantities corresponding to sizes when the radio fieldintensity is 100. This correspondence table can be formed for eachobject and stored in the database 104, and can also be formed bygeneralizing all objects and stored in the characteristic quantityadjusting unit.

FIGS. 11A and 11B show practical examples of the above processing.Referring to FIGS. 11A and 11B, an object information acquisition system300 has a characteristic quantity adjusting unit (not shown) in additionto an image sensing unit 101, RFID reader 102, image processor 103,database 104, characteristic quantity comparator 105, and display unit106 in the first embodiment.

FIGS. 11A and 11B illustrate cases in each of which an RFID tag 20 isread while the distance between an object 310 to which the RFID tag 20is attached and the object information acquisition system 300 ischanged. As shown in FIG. 11A, if radio field intensity 340 from theRFID tag 20 is low, the object 310 is far from the RFID reader 102.Therefore, if the image sensing unit 101 is installed adjacent to theRFID reader 102, the object 310 is far from the image sensing unit 101as well. That is, the object is presumably sensed in a relatively smallsize in an image, so the characteristic quantity of the object 310obtained from the image is also small. Accordingly, the reduction ratioof the characteristic quantity of the object is obtained from the radiofield intensity and correspondence table, a characteristic quantity 312of the object 310 read out from the database 104 is reduced to obtain acharacteristic quantity 313, and the characteristic quantity 313 iscompared with the characteristic quantity of an object 311 sensed in asmall size. This facilitates the comparison. On the other hand, if theradio field intensity from the RFID tag 20 is high, as shown in FIG.11B, the object is sensed in a large size in the sensed image.Therefore, the characteristic quantity 312 of the object 310 read outfrom the database 104 is enlarged to form a characteristic quantity 314,and the characteristic quantity 314 is compared with the characteristicquantity of the object 311 sensed in a large size.

As described above, it is possible, by enlarging or reducing thecharacteristic quantity in accordance with the radio field intensity, toincrease the accuracy of detection of the position of an object in animage, or reduce the processing load of an arithmetic unit.

Third Embodiment

The third embodiment according to the present invention will bedescribed in detail below with reference to the accompanying drawings.

FIG. 13 is a block diagram of an object information acquisition systemaccording to the third embodiment. Referring to FIG. 13, an objectinformation acquisition system 400 has an information storage unit 107for storing combined data, in addition to an image sensing unit 101,RFID reader 102, image processor 103, database 104, characteristicquantity comparator 105, and display unit 106. As the informationstorage unit 107, it is possible to use various storage media such as aRAM, flash ROM, hard disk, memory card, USB memory, magnetic tape,optical disk, magnetic disk, and various DVD disks. The combined data ismade up of image data obtained by the processing in the first or secondembodiment, position information of an object in an image, andinformation unique to the object acquired from the database 104. Theimage data is stored in the form of a moving image or still image. It isalso possible to store sensing condition information such as the time ofsensing, and location information obtained from a system such as GPS.

In the first to third embodiments described above, when RFID tags havingno special function are to be read, the image sensing unit senses animage and at the same time the RFID reader reads RFID tags within theimage sensing range. The characteristic quantity information of anobject corresponding to ID data of the read RFID tag is acquired fromthe database, and compared with characteristic quantities obtained byprocessing the sensed image, thereby finding matching portions.Therefore, even when a plurality of RFID tags is simultaneously read, itis possible to determine an object from which information is read.

Also, by displaying the information of an RFID tag in a position wherethe RFID tag exists in the sensed image, it is possible to provide anRFID read system which is easy to see by a human and widely applicable.

Furthermore, if an RFID tag is located in a position invisible from thereader or if an object is partially obstructed, the object can bedetected, and its existence position can be determined. In addition,before characteristic quantities are compared, the characteristicquantities of an object are enlarged or reduced on the basis of theradio field intensity. As a consequence, the load of the process ofcomparison with the characteristic quantities of a sensed image can bereduced.

Fourth Embodiment

The fourth embodiment according to the present invention will bedescribed in detail below with reference to the accompanying drawings.

If RFID tags described above can be attached to (adhered to,incorporated in, or embedded in) all objects, these objects can beeasily identified by using the RFID reader.

Since, however, RFID tags and objects are originally different things,an RFID tag can be intentionally detached (peeled off or removed) froman object and singly used. In particular, a non-contact IC card (e.g.,an ID card or entrance card) carried by a person can be extremely easilyused by any third person. Also, an RFID tag may be detached from anobject by some accident.

In addition, the RFID reader cannot always read RFID tags with 100%reliability. For example, the read accuracy may change owing to thepositional relationship (the distance or direction) between a tag andthe reader. Also, the read accuracy often largely decreases if aplurality of RFID tags overlaps each other, or if a conductor such as ametal is present around an RFID tag.

Accordingly, no high reliability can be obtained even when an object isidentified or authenticated by using an RFID tag alone.

To solve this problem, Japanese Patent Laid-Open No. 2000-261751 (priorart 3), for example, discloses an image authentication system which is acombination of a camera and RF receiver (RFID reader). An RFID tag isread simultaneously with camera sensing, and combined data is generatedby integrating the sensed image data and the read identifier. Thisassures that the image data and identifier are acquired at the sametime. Prior art 3 claims that the user can confirm the existence of apredetermined object by viewing the image data, and can confirm that theidentifier read simultaneously with the image data is a valid oneattached to the object.

Unfortunately, this image authentication system described in prior art 3does not well take account of a case in which a plurality of objectshaving RFID tags attached exists near the system. For example, ifobjects A, B, C, . . . , are arranged near each other, an image sensedby the camera described in prior art 3 may contain a plurality ofobjects.

Also, when an RFID tag of an object is read by using the RF receiver(RFID reader) described in prior art 3, it is highly likely to readidentifiers ID_A, ID_B, ID_C, . . . , attached to objects A, B, C, . . ..

In addition, prior art 3 describes the use of a directional antenna asan antenna of the RF receiver. However, the direction range (withinwhich RFID tags can be read) of this directional antenna cannot becontrolled. Therefore, it is highly likely that all RFID tags arrangedbefore the RF receiver are read, although there is no possibility thatRFID tags arranged behind the RF receiver are read.

That is, when the system described in prior art 3 is used, an imagecontaining objects A, B, C, . . . , and identifiers ID_A, ID_B, ID_C, .. . , are simultaneously read out, so the correspondence of each objectcontained in the image to its identifier cannot be known. Furthermore,even though the number of objects contained in the image is m (m is 1 ormore), the number of read identifiers may be n (n is equal to or largerthan m). This makes it more difficult to make a plurality of objects inan image correspond to a plurality of identifiers.

The fourth embodiment has been made in consideration of the aboveproblems, and has as its object to match a sensing range with an RFIDread range by controlling sensing parameters and RFID read parameters bysynchronizing them with each other, when an RFID tag attached to acertain object is to be read while the object is being shot.

It is another object of the fourth embodiment to always shoot only oneobject and read its RFID tag by controlling sensing parameters and RFIDread parameters by synchronizing them with each other, such that thenumber of objects contained in the sensed image or the number of readidentifiers is one.

It is still another object of the fourth embodiment to find only apredetermined object by storing combined data made up of image data andan identifier, only when a read identifier partially or entirely matchesa predetermined identifier, and store the time and location of findingto use them in tracking or managing the object.

The fourth embodiment will be explained by taking, as an example of anexternal apparatus, an object information acquisition apparatus whichsenses an object by a camera function, and simultaneously reads theinformation (identifier) of an RFID tag attached to the object by anRFID read function.

FIG. 14 is a block diagram showing an example of the arrangement of theobject information acquisition apparatus according to the fourthembodiment. As shown in FIG. 14, an object information acquisitionapparatus 500 includes an image sensing unit 510, RFID reader 520,parameter controller 530, combined data generator 540, operation unit550, display unit 560, and storage unit 570. Details of theseconstituent elements will be explained in order below.

The image sensing unit 510 has a predetermined zooming mechanism 511,and can control the zooming magnification or the viewing angle and focaldistance (focusing). Preferably, the image sensing unit 510 has apan/tilt mechanism (not shown), and can change the direction of sensingin the horizontal and vertical directions.

The RFID reader 520 has a directional antenna 521 for communicating byradio only with RFID tags in a specific direction. The RFID reader 520can control the range (to be referred to as an antenna direction rangehereinafter) within which a radio field emitted from the directionalantenna 521 radiates, and the antenna output level (the radiantintensity or reception intensity of an electromagnetic wave) havinginfluence on the longitudinal coverage of the radio field.

The parameter controller 530 simultaneously controls sensing parametersof the image sensing unit 510 and RFID read parameters of the RFIDreader 520 by synchronizing them with each other, in accordance withuser's instructions input from the operation unit 550. The method ofthis synchronous control will be described later with reference to FIGS.16 and 17. Note that instructions from the user are not alwaysnecessary, and the parameter controller 530 can automatically performsynchronous control of parameters in accordance with predeterminedcontrol rules.

The combined data generator 540 receives the sensed image data from theimage sensing unit 510, and also receives the identifier of the readRFID tag from the RFID reader 520. After that, the combined datagenerator 540 generates combined data by combining the image data andidentifier by a predetermined method. During this generation, it is alsopossible to perform predetermined conversion, e.g., encryption or datacompression, on the combined data.

The display unit 560 receives the image data and identifier from thecombined data generator 540, and displays, to the user, these data bymethods respectively suited to the data.

That is, the image data is displayed as a bitmap, and the identifier isdisplayed as characters or as a bitmap after being converted into anicon or graph by using a predetermined correspondence table.

Note that if the object information acquisition apparatus 500 isautomatically or remotely controlled, the operation unit 550 and displayunit 560 may be omitted.

The storage unit 570 stores the combined data received from the combineddata generator 540 into a storage medium. As the storage medium, it ispossible to use various media such as a RAM, flash ROM, hard disk,memory card, USB memory, magnetic tape, optical disk, magnetic disk, andvarious DVD disks.

FIG. 15 is a view showing an example of the arrangement of thedirectional antenna 521 having controllable characteristics. As shown inFIG. 15, the directional antenna 521 is a patch array antenna 522 inwhich a large number of patch antennas 523 are arranged into an array.Each patch antenna 523 is a small square, and a total of 16×16=256antennas are arranged. However, the number of elements can be freelyincreased or decreased to some extent, and has influence on the antennacharacteristics.

Note that the directivity can be increased by increasing the number ofelements not only in the patch array antenna 522 but also in any generalarray antenna. Note also that optimum values of the element size and theinterval between the elements are determined by the carrier frequency ofthe radio field transmitted and received by the antenna.

In the patch array antenna 522 according to this embodiment, anelectrical switch (not shown) is inserted between each element and afeeder wire so as to feed only elements within a predetermined rangefrom the center of the antenna. This electrical switch can beimplemented by a transistor or an MEMS (Micro Electro-Mechanical System)switch. A feeding range 524 indicates a range surrounding only fedelements.

FIG. 16 is a view showing a practical example of the correspondence ofthe zooming magnification of sensing as one sensing parameter to thefeeding range of the antenna as one RFID read parameter. A sensing rangewhen the zooming magnification of sensing is changed can beautomatically measured by viewing the sensed image with human eyes or byimage processing. Also, the sharpness (direction range) of thedirectivity of the radio field when the feeding range in the X and Ydirections of the antenna is changed can be calculated by measuring thereceived radio field intensity at many points in a radio field darkroom.

By measuring the sensing range and radio field direction rangebeforehand as described above, it is possible to select a plurality ofcorrespondences of the zooming magnification of sensing to the feedingrange of the antenna by which the sensing range and radio fielddirection range substantially match. Consequently, the correspondencesas shown in FIG. 16 can be obtained.

Note that the numerical values indicated in the columns of the feedingrange in the X and Y directions of the antenna correspond to the X- andY-coordinates shown in FIG. 15 when the interval of the patch antennas523 is “1”.

FIG. 17 is a view showing a practical example of the correspondence ofthe focal distance (F-number) of sensing as one sensing parameter to theoutput level of the antenna as one RFID read parameter. The radio fieldlongitudinal coverage when the output level of the antenna is changedcan be calculated by measuring the received radio field intensity atmany points within the direction range of the antenna in a radio fielddark room.

Accordingly, the correspondences as shown in FIG. 17 can be obtained byselecting the output level of the antenna such that the focal distance(F-number) of sensing matches the radio field longitudinal coverage ofthe antenna.

Note that the numerical values in the columns shown in FIGS. 16 and 17are examples, and change in accordance with, e.g., the part arrangement,structure, and manufacturing method of the object informationacquisition apparatus 500. Therefore, it is preferable to performmeasurements for each type of apparatus, and form differentcorrespondences on the basis of the measurement results.

In the above arrangement, the parameter controller 530 controls theimage sensing unit 510 and RFID reader 520 by synchronizing them witheach other, so that a sensing range 11 in which the image sensing unit510 senses an image and an RFID read range 12 in which the RFID reader520 reads an RFID tag are substantially the same. In this way, only oneobject B can be contained in the sensing range 11 and RFID read range12.

Control executed by the parameter controller 530 when the objectinformation acquisition apparatus 500 senses a desired object and at thesame time reads the identifier of an RFID tag adhered to the object willbe explained below. This control will be explained by taking, as anexample, a case in which the object B shown in FIG. 14 is sensed as adesired object.

FIG. 18 is a flowchart showing the control sequence of the parametercontroller 530 according to the fourth embodiment. First, in step S1801,the parameter controller 530 instructs the image sensing unit 510 tosense the object B. Accordingly, the image sensing unit 510 adjusts thezooming magnification and focal distance of the zooming mechanism 511such that the sensing range equals the sensing range 11 of the object B,and senses only the object B.

Then, in step S1802, the parameter controller 530 receives the zoomingmagnification and focal distance as sensing parameters from the imagesensing unit 510, and, on the basis of these sensing parameters, theparameter controller 530 determines the read parameters of the RFIDreader 520 such that the sensing range 11 and the RFID read range 12 aresubstantially the same.

More specifically, if the zooming magnification of sensing is 1.5, theparameter controller 530 determines+5 or less (the feeding range 524shown in FIG. 15) as the feeding range in the X and Y directions fromthe correspondence of the zooming magnification to the feeding range ofthe antenna shown in FIG. 16. If the focal distance of sensing is 4.3,the parameter controller 530 determines 90% as the output level from thecorrespondence of the focal distance to the output level of the antennashown in FIG. 17.

In step S1803, the parameter controller 530 designates the determinedfeeding range and output level to the RFID reader 520. Accordingly, theRFID reader 520 controls the directional antenna 521 in accordance withthe feeding range and output level, and reads the identifier of the RFIDtag adhered to the object B within the RFID read range 12.

In this manner, the image data obtained by sensing only the object B andthe identifier read from the RFID tag adhered to the object B are inputto the combined data generator 540. The combined data generator 540generates combined data from the image data and identifier, and candisplay the combined data on the display unit 560 or store it in thestorage unit 570.

That is, the combined data generator 540 generates combined data onlywhen one object alone is sensed or one identifier alone is read. Thisimproves the use value of the combined data. Note that if a plurality ofobjects is sensed, it is also possible to perform sensing again.

Also, the combined data generator 540 determines, by using an identifierdetermination unit (not shown), whether the read identifier partially orentirely matches a predetermined identifier, and, only when the twoidentifiers are found to match each other, stores combined dataincluding the image data and identifier into the storage unit. As aconsequence, only a predetermined object can be found.

Furthermore, the time of finding, the location of finding, and the likeof a predetermined identifier are stored together with the combined datainto the storage unit 570, and can be used to track the object or managethe storage location.

In this case, the time of finding of a predetermined identifier isacquired by using a system timer (not shown). Also, the location offinding of a predetermined identifier is the installation location ofthe object information acquisition apparatus which the user inputs fromthe operation unit 550 and stores in the storage unit 570 in advance, oris acquired by using a position measurement system such as a GPS (GlobalPositioning System).

Fifth Embodiment

The fifth embodiment according to the present invention will bedescribed in detail below with reference to the accompanying drawings.

In the fifth embodiment, an object authentication apparatus which shootsan object by a camera function, and at the same time authenticates theobject by reading the information (identifier) of an RFID tag adhered tothe object by an RFID read function will be explained.

FIG. 19 is a block diagram showing an example of the arrangement of theobject authentication apparatus according to the fifth embodiment. Notethat the same reference numerals as in FIG. 14 denote the samecomponents in FIG. 19, and an explanation thereof will be omitted.Components different from FIG. 14 will be explained below.

As shown in FIG. 19, an object authentication apparatus 600 of the fifthembodiment additionally has an image characteristic quantity extractionunit 610, combined data storage unit 620, and authentication unit 630.

The image characteristic quantity extraction unit 610 receives sensedimage data from an image sensing unit 510, and automatically extractsimage characteristic quantities such as the shapes and colors of sensedobjects by using a predetermined image processing algorithm.

A combined data generator 540 receives the image characteristicquantities from the image characteristic quantity extraction unit 610,and also receives the identifiers of read RFID tags from an RFID reader520. After that, the combined data generator 540 generates combined databy combining the image characteristic quantities and identifiers by apredetermined method.

The combined data storage unit 620 stores a list of combined data madeup of image characteristic quantities and identifiers obtained fromvarious objects in advance.

That is, image sensing is performed for each of various objectsbeforehand, image characteristic quantities are extracted, and at thesame time identifiers are read from RFID tags. Combined data are formedby combining the extracted image characteristic quantities and the readidentifiers, and a list of the combined data is stored.

The authentication unit 630 collates the combined data received from thecombined data generator 540 with the combined data list stored in thecombined data storage unit 620, and authenticates an object. Thisauthentication is performed by two stages, i.e., the collation ofidentifiers and the collation of image characteristic quantities. Thismakes the reliability much higher than that of authentication of onlyidentifiers performed using the conventional RFID technique orauthentication of only face images or fingerprint images performed usingthe conventional biometrics technique.

In the above arrangement, a parameter controller 530 controls the imagesensing unit 510 and RFID reader 520 by synchronizing them such that asensing range 11 in which the image sensing unit 510 senses an image andan RFID read range 12 in which the RFID reader 520 reads an RFID tag aresubstantially the same. The image characteristic quantity extractionunit 610 receives shot image data from the image sensing unit 510, andautomatically extracts the image characteristic quantities such as theshape and color of a sensed object by using a predetermined imageprocessing algorithm. The combined data generator 540 generates combineddata by combining the image characteristic quantities from the imagecharacteristic quantity extraction unit 610 and the identifier of anRFID tag from the RFID reader 520 by a predetermined method. Theauthentication unit 630 collates the combined data received from thecombined data generator 540 with the combined data list stored in thecombined data storage unit 620, thereby authenticating an object. Theauthentication result (authentication success or authentication failure)is displayed on a display unit 560.

Accordingly, when only one object is sensed and only one identifier isread, the combined data generator 540 generates combined data, and theauthentication unit 630 authenticates the object. This improves the usevalue of the combined data.

The fourth and fifth embodiments can prevent read of RFID tags otherthan a sensed object.

The present invention has been described above by way of its preferredembodiments. However, the present invention is not limited to the aboveembodiments, but can be variously modified within the range described inthe scope of claims.

CLAIM OF PRIORITY

This application claims priority from Japanese Patent Application Nos.2004-220683 filed on Jul. 28, 2004 and 2004-220381 filed on Jul. 28,2004, which are hereby incorporated by reference herein.

1. An information acquisition apparatus having a sensing deviceconfigured to sense an object, and a reading device configured to readinformation of a tag attached to the object, comprising: an extractingdevice configured to extract information concerning a characteristic ofan object in an image sensed by said sensing device; and adiscrimination device configured to discriminate a position of theobject corresponding to the information of the tag read by said readingdevice, on the basis of the information concerning the characteristic ofthe object in the sensed image extracted by said extracting device andthe information of the tag read by said reading device.
 2. The apparatusaccording to claim 1, further comprising a storage device configured tostore the information of the tag attached to the object andpredetermined information and the information concerning thecharacteristic of the object, such that the information of the tagcorresponds to the information of the object, wherein saiddiscrimination device performs the discrimination on the basis of theinformation stored in said storage device.
 3. The apparatus according toclaim 1, further comprising a display device configured to display theobject in the sensed image such that the object is discriminable by thepredetermined information thereof, on the basis of the discrimination bysaid discrimination device.
 4. The apparatus according to claim 1,wherein said extracting device extracts the information concerning thecharacteristic of the object in the sensed image by predetermined imageprocessing.
 5. The apparatus according to claim 4, wherein theinformation concerning the characteristic of the object is at least oneof an edge, a color, a luminance, a texture, the presence/absence of amotion, a shape, and a distance between characteristic points.
 6. Theapparatus according to claim 1, wherein the predetermined information ofthe object is one of information unique to the object and one of a nameand model number of the object.
 7. The apparatus according to claim 3,wherein said display device makes discrimination possible bysuperimposing, on the object in the sensed image, the predeterminedinformation for identifying the object.
 8. The apparatus according toclaim 3, wherein said display device comprises a designating deviceconfigured to designate switching between the predetermined informationof the object, and switches display to another predetermined informationwhen switching is designated by said designating device.
 9. Theapparatus according to claim 1, wherein on the basis of a radio fieldintensity which changes in accordance with a distance between saidreading device and the tag of the object, the information concerning thecharacteristic of the object in the sensed image is enlarged or reduced.10. The apparatus according to claim 1, wherein said reading devicecomprises an antenna having directivity corresponding to a sensing rangeof said sensing device.
 11. The apparatus according to claim 1, furthercomprising a control device configured to control a sensing parameter ofsaid sensing device and a read parameter of said reading device, suchthat a sensing range in which said sensing device senses an objectmatches a read range in which said reading device reads information of atag.
 12. The apparatus according to claim 11, wherein said readingdevice comprises a directional antenna having directivity within apredetermined range centering around a viewing direction of said sensingdevice, a direction range of said directional antenna is controllable asthe read parameter, and a viewing angle or zooming magnification iscontrollable as the sensing parameter, and said control device controlsthe direction range of said directional antenna by synchronizing thedirection range with the viewing angle or zooming magnification.
 13. Theapparatus according to claim 11, wherein said reading device comprises adirectional antenna having directivity within a predetermined rangecentering around a viewing direction of said sensing device, a focaldistance is controllable as the sensing parameter, and an antenna outputof said directional antenna is controllable as the read parameter, andsaid control device controls the antenna output by synchronizing theantenna output with the focal distance.
 14. An information acquisitionapparatus processing method comprising: an extraction step of extractinginformation concerning a characteristic of an object in a sensed image;an identification step of identifying information of a tag attached tothe object in the sensed image; and a discrimination step ofdiscriminating a position of the object corresponding to the informationof the tag identified in the identification step, on the basis of theinformation concerning the characteristic of the object in the sensedimage extracted in the extraction step, and the information of the tagidentified in the identification step.
 15. A program which causes acomputer to execute the steps of an information acquisition apparatusprocessing method cited in claim
 14. 16. A computer-readable recordingmedium which records a program cited in claim 15.